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Algorithms  motivated  by  simiiar  processing  in 
animais  and  humans: 

-  Hearing  and  sound  classification 

-  Vision  and  identification  of  objects 

Text-independent  robust  speaker  identification 

-  Identifying  the  speaker  from  the  “music”  of  his  voice 

Speaker-independent  speech  recognition 

-  Identifying  phonemes,  vowels,  words  from  their  inherent  sounds 

Identification  of  musical  Instruments  (“timbre”) 
Applications  to  acoustic  signal  recognition 

-  Fault  identification  in  tools  and  wear  prediction 

-  Ground  vehicle  identification  from  array  microphones 

NEXT  CHALLENGE:  Biology  Inspired 
Sensor  Network  processing 
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Acoustic  Vehicle  Classification 
Objectives  and  Challenges 
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•  Develop  systematic  methodologies  and  algorithms;  not  ad  hoc 

•  Robust  Target  ID  (wrt  environment,  terrain,  speed) 

•  Algorithms  for  combined  DOA  (localization)  and  target  ID 

-  Localization  assisted  ID 

-  ID  assisted  localization 

•  Multi-target  detection,  ID  and  DOA;  separation  of  closely  spaced 
targets 

•  Robust  feature  extraction  from  auditory  models;  dynamic  DOA 
and  ID 

•  Algorithm  evaluation  in  the  field  and  comparison  against 
conventional  algorithms  for  detection,  DOA  and  ID 
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Mu/f/reso/uf/on  Adaptive  Acoustic 

Ciassification 

•  ArcNtecture 
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Data 
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Multiresolution 

Nonlinear 

Learning  Clustering 

Preprocessor 
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Postprocessor 
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•  Architecture  and  formulation  address  two  most  important  issues: 

-  Progressive  classification;  Which  features  to  use  and  when 

-  Efficient  design  of  databases  for  reference  signals  and  fast  search 

•  Trade-off  between  efficiency  in  features  (compression)  and 
accuracy  in  classification  leads  to 

•  Mathematical  formulation  of  the  problem: 

-  Combined  compression  and  classification  for  general  signals 

-  Content-based  feature  extraction  and  use  for  classification 
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Multiresolution  Preprocessors 
Auditory  Filtering 


Two  auditory  filters,  motivated  and  designed  according  to  acoustic 
physioiogy  and  acoustic  cortex  modeis,  were  used  to  compute  the 
timbre  spectrogram  of  one  particuiar  subframe  in  each  frame 

Time  Waveform 
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The  first  filter  mimics  the  action  of  the  inner  ear 

Computes  the  spectrogram  of  the  sound  sample,  and  performs 
various  nonlinear  operations,  which  models  the  nonlinear 

fluid-cilia  couplings  and  ionic  channels  of  conduction 

r,  n.  ( Wavolet  T ransf orm  ) 
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Multiresolution  Preprocessor: 
Auditory  Filtering 

Multiresolution  cortical  filter  outputs 
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•  The  second  filter  models  the  multiscale  processing  of  the 
signal  that  happens  in  the  auditory  cortex 

•  A  Ripple  Analysis  Model,  using  a  ripple  filter  bank,  acts 

on  the  output  of  the  inner  ear  to  give  multiscale  spectra  of 
the  sound  timbre  (Wavelet  Transform) 
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Postprocessor:  Multi  Resolution  (Wavelet) 
Tree  Structured  Vector  Quantization  (WTSVQ) 


Resolution  3 


Resolution  2 


Resolution  1 


First  perform  a 
multiresolution  wavelet 
representation  of  the  signals 

Consider  each  signal  f  at 
different  resolutions 

SOf,  SU,...,  S^*f 

Proceed  by  partitioning  the 
signal  space  at  various 
resolutions  in  progressively 
finer  cells 

Greedy  algorithm  works  by 
splitting  the  cell  with 
maximum  distortion  using 
finer  resolution  data 
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Layer  in  tree  I  =  J*-  m,  m  the  scale  ( top  layer  0:  coarsest) 
Cell  labels:  (layer,  index)  or  (scale,  index) 
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Can  we  mimic  and  understand 
the  abiiity  of  humans  to  do  partiai 
recognition  of  musicai  instruments 
and  DOA  in  a  combined  and 
mutuaiiy  enhancing  fashion? 


clarhiet 


Combine  the  Stereausis  model  and  its  derivatives ,  with  the 
Auditory  filtering  multiscale  VQ  algorithms 

Using  the  cochlea,  cortical,  or  combined  spectra,  perform 
DOA  on  a  “per  frequency  band  basis” 

Combine  portions  of  spectra  according  to  DOA 

Use  the  multiscale  classifier  to  ID  portions  of  spectra 
tagged  by  angle,  as  compared  to  stored  vehicle  spectra 

Repeat  the  cycle  as  the  scenario  evolves 
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gv1a1012.mat: 


Auditory  Processing  of 
Vehicie  Acoustic  Signals:  Cochlea 

type  1  speed  5  desert  gv1b2021  .mat:  type  1  speed  10  arctic 
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Auditory  processing  for  vehicie  signais  (cochiear  fiiter  banks) 

Left:  vehicie  type  1,  speed  5km/hr.  Right:  vehicie  type  1,  speed  10km/hr 
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Auditory  Processing  of 
Vehicie  Acoustic  Signais:  Cortex 
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Example  of  multi-resolution  representation  from 
cortical  module 
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Stereausis  Output  for  Two  Vehicles 


DO  A  pattern  for  2  vehicles 


time  (in  second) 


Relatively  easy  case:  Large  angular  separation  between  two  vehicles 
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Leaf  Node  Entropies  for 
PTSVQ  Tree  of  Vehicle  Type  8 


cell  entropy 
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Options  in  Appiying  WTSVQ 
to  Acoustic  Vehicie  Ciassification 


GTSVQ:  A  global  tree-structured  multi-resolution  clustering 
mechanism  that  mimics  the  aggressive  and  topological  hearing 
capabilities  of  biologicai  systems.  Here  a  globai  tree  is  built  on 
training  data  from  ali  vehicles.  New  vehicle  insertion  problem. 

LVQ:  A  supervised  learning  neural  network,  LVQ  achieves  optimal 
classification  in  the  Bayes  sense.  It  has  the  disadvantages  of  a 
long  search  time  and  sensitivity  to  initial  conditions. 

Paraliel  TSVQ  (PTSVQ):  build  one  (or  more)  trees  for  each  vehicle. 
It  achieves  a  trade-off  between  GTSVQ  and  LVQ  on  ciassification 
performance  and  search  time.  Easy  new  vehicle  insertion. 

The  foliowing  node  aliocation  schemes  are  examined  for  PTSVQ: 

-  PTSVQ(1):  Allocation  based  on  sample  a  priori  probability 

-  PTSVQ(2):  Allocation  based  on  equal  distortion 

-  PTSVQ(3):  Allocation  according  to  vehicle  speed 
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Performance  Comparisons 

\ 

among  Options 

asR 

L  V  -  n  f\  r>k 

I  ^  J  ^ 

VV  ^  " 

Center  for  Auditory 
and  Acoustic  Research 


classification  rate  v.s.  tree  size 


Classification  Performance:  70%  sampies  for 
training,  30%  for  testing  (same  microphone) 
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Tagging  Portions  of  Spectra 
Based  on  “per  Band”  DOA  Estimates 


*  Angular  position  of  ach  peak  corresponds  to  DOA  estimate 
from  each  cochlea  band 

*  Can  use  up  to  128  bands 

*  Amplitude  indicates  signal  energy  in  the  band 


400 


*  Low  pass  filtering  is  performed  on  groups  of  band 
amplitudes  and  the  resulting  peak  is  used  as  the  DOA 
estimate  for  the  vehicie 

*  Cluster  according  to  angular  position  of  peaks:  spectral 
portions  tagged  by  angle 


^  -  r\.  r\ 


\j 


Center  for  Auditory 
and  Acoustic  Research 


Page  15 


